Automated data extraction from research texts has been steadily improving, with the emergence of large language models (LLMs) accelerating progress even further. Extracting data from plots in research papers, however, has been such a complex task that it has predominantly been confined to manual data extraction. We show that current multimodal large language models, with proper instructions and engineered workflows, are capable of accurately extracting data from plots. This capability is inherent to the pretrained models and can be achieved with a chain-of-thought sequence of zero-shot engineered prompts we call PlotExtract, without the need to fine-tune. We demonstrate PlotExtract here and assess its performance on synthetic and published plots. We consider only plots with two axes in this analysis. For plots identified as extractable, PlotExtract finds points with over 90% precision (and around 90% recall) and errors in x and y position of around 5% or lower. These results prove that multimodal LLMs are a viable path for high-throughput data extraction for plots and in many circumstances can replace the current manual methods of data extraction.

Ideal for healthcare, finance, legal and enterprise organizations, etherFAX AI uses Microsoft's Cognitive Services to extract and digitize data from a range of unstructured documents and forms, streamlining workflow and eliminating manual entry. "Powerful AI document data extraction transforms unstructured documents -- such as faxes, PDFs, and paper-based forms -- into structured, searchable data ready to integrate into workflow processes and applications," said Ben Manning, Director of Product at etherFAX. "Utilizing these cognitive services for data extraction turns content locked in unstructured forms into usable, structured data and fields that can be used to automate workflows and eliminate manual processes." Manually searching and keying in data into fields is time-consuming and often leads to human error. With etherFAX's AI solutions, organizations can quickly transform content locked in unstructured documents such as PDFs and paper-based forms into structured, searchable data ready to be integrated into workflows, applications, or EMRs.